One proposed structure of a heat recovery system has a heat exchange path arranged to perform heat exchange between emission of an internal combustion engine and a heat exchange medium, a bypass path arranged to discharge the emission with bypassing the heat exchange path, and a heat insulating layer coaxially arranged with the heat exchange path and the bypass path and formed between the heat exchange path and the bypass path (see, for example, Japanese Patent Laid-Open No. 2007-32561). In this prior art heat recovery system, the heat insulating layer is formed around the outer circumference of the bypass path, and the heat exchange path is formed around the outer circumference of the heat insulating layer. A heat exchange medium flow path is further formed around the outer circumference of the heat exchange path. A valve is provided in the bypass path to shut off the bypass path. During warm-up of the internal combustion engine, the valve provided in the bypass path is closed to shut off the bypass path. In this state, the emission of the internal combustion engine flows through the heat exchange path. The heat exchange medium is heated by the heat exchange with the emission flowing through the heat exchange path and is used to heat up the cooling water flowing through the internal combustion engine. This accelerates the warm-up of the internal combustion engine. After completion of the warm-up of the internal combustion engine, the valve provided in the bypass path is opened to release the shutoff of the bypass path. In this state, the emission of the internal combustion engine flows through the bypass path and is discharged to the downstream. In the heat recovery system of this prior art structure, the heat insulating layer prevents the heat exchange medium from being overheated.